Mark Giampapa et al. (Mark Giampapa, Thomas Gooding, Todd Inglett, Robert W. Wisniewski. Experiences with a lightweight supercomputer kernel: Lessons learned from Blue Gene's CNK. Proceedings of the 2010 ACM/IEEE International Conference for High Performance Computing (SC10). New Orleans, La. November 2010) outlines the different approaches taken by HPC operating system kernels. The kernels fall into a spectrum defined at one end by Light-Weight Kernels (LWKs) and at the other end by Full-Weight Kernels (FWKs). Typically, LWKs such as Compute Node Kernel (CNK) provide higher performance and scalability than FWKs such as Linux but less functionality. LWKs achieve higher performance and scalability because they give applications direct access to physical resources. FWKs maintain ownership of physical resources and arbitrate access among many applications. LWKs have less functionality than FWKs because LWKs are specialized. Linux runs on many different platforms and has a large developer base.
There have been attempts to bridge the gaps between LWKs and FWKs. These approaches may fall into two categories: (1) add FWK functionality to an LWK; and (2) improve FWK performance for HPC applications. Application performance has many aspects including efficiency, scalability, and reliability. In the present disclosure, performance refers to overall application performance, measured by time to completion.
CNK, which runs on the Blue Gene® supercomputer from International Business Machines Corporation (IBM®), adds FWK (e.g., Linux) functionality to the HPC. This approach has been successful in adding Linux functionality while maintaining performance and scalability, but functionality gaps still exist, e.g., file mmap is read-only, fork is not supported, and thread scheduling is non-preemptive.
The Pallacios/Kitten project is another approach (See, John Lange, Kevin Padretti, Trammell Hudson, Peter Dinda, Zheng Cui, Lei Xia, Patrick Bridges, Andy Gocke, Steven Jaconette, Mike Levenhagen, Ron Brightwell. Palacios and Kitten: New high performance operating systems for scalable virtualized and native supercomputing. Proceedings of the 24th IEEE International Parallel and Distributed Processing Symposium (IPDPS 2010). Atlanta, Ga. April, 2010). Kitten is an LWK developed at Sandia®. Pallacios is a virtual machine monitor (VMM) that has been ported to Kitten. Pallacios allows both LWKs and FWKs to run unmodified on Kitten; it allows Kitten to host Linux as a guest operating system. This approach provides the performance and scalability of LWKs and the functionality of FWKs. However, the functionality of the FWK is limited by the VMM. For example, Pallacios does not support multi-core guests at this time. Further, VMM is a complex layer of software requiring maintenance for each Linux release.
ZeptoOS modifies Linux to set aside memory during boot and to allocate it in large regions (See e.g., Kazutomo Yoshii, Kamil Iskra, P. Chris Broekema, Harish Naik, Pete Beckman. Characterizing the performance of Big Memory on Blue Gene Linux. Proceedings of the 22nd International Workshop on Parallel Programming Models and Systems Software for High-End Computing (P2S2 2009). Vienna, Austria. September, 2009). This memory management modification along with removing unnecessary daemons to reduce noise improved the performance and scalability of Linux. Schmueli took a similar approach by adding large page support to Linux (See, Edi Shmueli, George Almási, Jose Brunheroto, Jose Castaños, Gabor Dózsa, Sameer Kumar, Derek Lieber. Evaluating the effect of replacing CNK with Linux on the compute-nodes of Blue Gene/L. Proceedings of the 22nd ACM International Conference on Supercomputing (ICS 2008). Island of Kos, Greece. June 2008). In addition to removing unnecessary daemons, running daemons were synchronized to further reduce noise.
The ZeptoOS approach and the Schmueli approach improve the performance and scalability of Linux. However, both approaches require significant changes to Linux, and the changes must be manually merged to each new release of the Linux kernel.
The Cell Broadband Engine (Cell/BE) consists of a general purpose PowerPC called the PPE (PowerPC Processing Element) and eight vector processors called the Synergistic Processing Elements (SPEs). The PPE and SPEs do not share memory, and each SPE has its own local memory. Linux runs on the PPE. It has been modified to provide access to SPE state and memory through a file system interface. System calls are added to allocate an SPE and to start an SPE. Application code that is run on the SPE must not have any system call instructions; rather, system call instructions are replaced by code that returns code to Linux running on the PPE.